Skip Headers

Oracle® Enterprise Manager Advanced Configuration 10g Release 2 (10.2) Part Number B16242-02 |
Home | Book List | Contents | Index | Master Index | Feedback

Previous | Next
| View PDF

# 8 Maintaining and Troubleshooting the Management Repository

This chapter describes maintenance and troubleshooting techniques for maintaining a well-performing Management Repository.

Specifically, this chapter contains the following sections:

- Management Repository Deployment Guidelines

- Management Repository Data Retention Policies

- Changing the SYSMAN Password

- Dropping and Recreating the Management Repository

- Troubleshooting Management Repository Creation Errors

- Improving the Login Performance of the Console Home Page

## 8.1 Management Repository Deployment Guidelines

To be sure that your management data is secure, reliable, and always available, consider the following settings and configuration guidelines when you are deploying the Management Repository:

- Install a RAID-capable Logical Volume Manager (LVM) or hardware RAID on the system where the Management Repository resides. At a minimum the operating system must support disk mirroring and stripping. Configure all the Management Repository data files with some redundant configuration.

- Use Real Application Clusters to provide the highest levels of availability for the Management Repository.

- If you use Enterprise Manager to alert administrators of errors or availability issues in a production environment, be sure that the Grid Control components are configured with the same level of availability. At a minimum, consider using Oracle Data Guard to mirror the Management Repository database. Configure the Data Guard environment for no data loss.

  See Also: Oracle High Availability Architecture and Best Practices Oracle Data Guard Concepts and Administration

- Oracle strongly recommends that archive logging be turned on and that a comprehensive backup strategy be in place prior to an Enterprise Manager implementation going live in a production environment. The backup strategy should include both incremental and full backups as required.

  See Also: Oracle Enterprise Manager Grid Control Installation and Basic Configuration for information about the database initialization parameters required for the Management Repository

## 8.2 Management Repository Data Retention Policies

When the various components of Enterprise Manager are configured and running efficiently, the Oracle Management Service gathers large amounts of raw data from the Management Agents running on your managed hosts and loads that data into the Management Repository. This data is the raw information that is later aggregated, organized, and presented to you in the Grid Control Console.

After the Oracle Management Service loads information into the Management Repository, Enterprise Manager aggregates and purges the data over time.

The following sections describe:

- The default aggregation and purging policies used to maintain data in the Management Repository.

- How you can modify the length of time the data is retained before it is aggregated and then purged from the Management Repository.

### 8.2.1 Management Repository Default Aggregation and Purging Policies

Enterprise Manager aggregates your management data by hour and by day to minimize the size of the Management Repository. Before the data is aggregated, each data point is stored in a raw data table. Raw data is rolled up, or aggregated, into a one-hour aggregated metric table. One-hour records are then rolled up into a one-day table.

After Enterprise Manager aggregates the data, the data is then considered eligible for purging. A certain period of time has to pass for data to actually be purged. This period of time is called the retention time.

The raw data, with the highest insert volume, has the shortest default retention time, which is set to 7 days. As a result, 7 days after it is aggregated into a one-hour record, a raw data point is eligible for purging.

One-hour aggregate data records are purged 31 days after they are rolled up to the one-day data table. The highest level of aggregation, one day, is kept for 365 days.

The default data retention policies are summarized in Table 8-1.

Table 8-1 Default Repository Purging Policies

Aggregate Level | Retention Time
--- | ---
Raw metric data | 7 days
One-hour aggregated metric data | 31 days
One-day aggregated metric data | 365 days

If you have configured and enabled Application Performance Management, Enterprise Manager also gathers, saves, aggregates, and purges response time data. The response time data is purged using policies similar to those used for metric data. The Application Performance Management purging policies are shown in Table 8-2.

Table 8-2 Default Repository Purging Policies for Application Performance Management Data

Aggregate Level | Retention Time
--- | ---
Raw response time data | 24 hours
One-hour aggregated response time data | 7 days
One-hour distribution response time data | 24 hours
One-day aggregated response time data | 31 days
One-day distribution aggregated response time data | 31 days

### 8.2.2 Management Repository Default Aggregation and Purging Policies for Other Management Data

Besides the metric data and Application Performance Monitoring data, other types of Enterprise Manager data accumulates over time in the Management Repository.

For example, the last availability record for a target will also remain in the Management Repository indefinitely, so the last known state of a target is preserved.

### 8.2.3 Modifying the Default Aggregation and Purging Policies

The Enterprise Manager default aggregation and purging policies were designed to provide the most available data for analysis while still providing the best performance and disk-space requirements for the Management Repository. As a result, you should not modify these policies to improve performance or increase your available disk space. Modifying these default policies can affect the performance of the Management Repository and have adverse reactions on the scalability of your Enterprise Manager installation.

However, if you plan to extract or review the raw or aggregated data using data analysis tools other than Enterprise Manager, you may want to increase the amount of raw or aggregated data available in the Management Repository. You can accomplish this by increasing the retention times for the raw or aggregated data.

To modify the default retention time for each level of management data in the Management Repository, you must insert additional rows into the MGMT_PARAMETERS table in the Management Repository database. Table 8-3 shows the parameters you must insert into the MGMT_PARAMETERS table to modify the retention time for each of the raw data and aggregate data tables.

Table names that contain "_RT_" indicate tables used for Application Performance Monitoring response time data. In the Table Name column, replace datatype with one of the three response time data types: DOMAIN, IP, or URL.

Table 8-3 Parameters for Modifying Default Data Retention Times in the Management Repository

Table Name | Parameter in MGMT_PARAMETERS Table | Default Retention Value
--- | --- | ---
MGMT_METRICS_RAW | mgmt_raw_keep_window | 7 days
MGMT_METRICS_1HOUR | mgmt_hour_keep_window | 31 days
MGMT_METRICS_1DAY | mgmt_day_keep_window | 365 days
MGMT_RT_METRICS_RAW | mgmt_rt_keep_window | 24 hours
MGMT_RT_datatype_1HOUR | mgmt_rt_hour_keep_window | 7 days
MGMT_RT_datatype_1DAY | mgmt_rt_day_keep_window | 31 days
MGMT_RT_datatype_DIST_1HOUR | mgmt_rt_dist_hour_keep_window | 24 hours
MGMT_RT_datatype_DIST_1DAY | mgmt_rt_dist_day_keep_window | 31 days

Note: If the first three tables listed in Table 8-3 are not partitioned, the Default Retention Value for each is 1, 7, and 31 days respectively, rather than the 7, 31, and 365 days listed for partitioned tables.

For example, to change the default retention time for the table MGMT_METRICS_RAW from seven days to 14 days:

1. Use SQL*Plus to connect to the Management Repository database as the Management Repository user.

   The default Management Repository user is `sysman`.

2. Enter the following SQL to insert the parameter and change the default value:

       INSERT INTO MGMT_PARAMETERS (PARAMETER_NAME, PARAMETER_VALUE)  VALUES ('mgmt_raw_keep_window','14');

Similarly, to change from the default retention time for all of the MGMT_RT_datatype_1DAY tables from 31 days to 100 days:

1. Use SQL*Plus to connect to the Management Repository database as the Management Repository user.

   The default Management Repository user is `sysman`.

2. Enter the following SQL to insert the parameter and change the default value:

       INSERT INTO MGMT_PARAMETERS (PARAMETER_NAME, PARAMETER_VALUE)  VALUES ('mgmt_rt_day_keep_window', '100');

### 8.2.4 Modifying Data Retention Policies When Targets Are Deleted

By default, when you delete a target from the Grid Control Console, Enterprise Manager automatically deletes all target data from the Management Repository.

However, deleting raw and aggregated metric data for database and other data-rich targets is a resource consuming operation. Targets can have hundreds of thousands of rows of data and the act of deleting this data can degrade performance of Enterprise Manager for the duration of the deletion, especially when several targets are deleted at once.To avoid this resource-consuming operation, you can prevent Enterprise Manager from performing this task each time you delete a target. When you prevent Enterprise Manager from performing this task, the metric data for deleted targets is not purged as part of target deletion task; instead, it is purged as part of the regular purge mechanism, which is more efficient.

In addition, Oracle strongly recommends that you do not add new targets with the same name and type as the deleted targets within 24 hours of target deletion. Adding a new target with the same name and type will result in the Grid Control Console showing data belonging to the deleted target for the first 24 hours.

To disable raw metric data deletion:

1. Use SQL*Plus to connect to the Management Repository as the Management Repository user.

   The default Management Repository user is SYSMAN. For example:

       SQL> connect sysman/oldpassword;

2. To disable metric deletion, run the following SQL command.

       SQL> EXEC MGMT_ADMIN.DISABLE_METRIC_DELETION();
       SQL> COMMIT;

To enable metric deletion at a later point, run the following SQL command:

1. Use SQL*Plus to connect to the Management Repository as the Management Repository user.

   The default Management Repository user is SYSMAN. For example:

       SQL> connect sysman/oldpassword;

2. To enable metric deletion, run the following SQL command.

       SQL> EXEC MGMT_ADMIN.ENABLE_METRIC_DELETION();
       SQL> COMMIT;

## 8.3 Changing the SYSMAN Password

The SYSMAN account is the default super user account used to set up and administer Enterprise Manager. It is also the database account that owns the objects stored in the Oracle Management Repository. From this account, you can set up additional administrator accounts and set up Enterprise Manager for use in your organization.

The SYSMAN account is created automatically in the Management Repository database during the Enterprise Manager installation. You also provide a password for the SYSMAN account during the installation.

See Also: Oracle Enterprise Manager Grid Control Installation and Basic Configuration for information about installing Enterprise Manager

If you later need to change the SYSMAN database account password, use the following procedure:

1. Shut down all the Oracle Management Service instances that are associated with the Management Repository.

   See Also: "Controlling the Oracle Management Service"

2. In the Grid Control Console, click the Targets tab, and then click All Targets on the sub tab.

3. Select the Management Services and Repository target and click Configure.

   Enterprise Manager displays the Monitoring Configuration page.

4. Enter the new password in the Repository password field and click OK.

   See Also: "Specifying New Target Monitoring Credentials"

5. Change the password of the SYSMAN database account using the following SQL*Plus commands:

       SQL>connect sysman/oldpassword;
       SQL>alter user sysman identified by newpassword;

6. For each Management Service associated with the Management Repository, locate the `emoms.properties` configuration file.

   The `emoms.properties` file can be found in the following directory of the Oracle Application Server Home where the Oracle Management Service is installed and deployed:

       IAS_HOME/sysman/config/

7. Locate the following entries in the `emoms.properties` file:

       oracle.sysman.eml.mntr.emdRepPwd=ece067ffc15edc4f
       oracle.sysman.eml.mntr.emdRepPwdEncrypted=TRUE

8. Enter your new password in the first entry and enter FALSE in the second entry.

   For example:

       oracle.sysman.eml.mntr.emdRepPwd=new_password
       oracle.sysman.eml.mntr.emdRepPwdEncrypted=FALSE

9. Save and exit the `emoms.properties` file and restart each Management Service associated with the Management Repository.

10. After the Management Service has started, check the contents of the `emoms.properties` file to be sure the password you entered has been encrypted.

    For example, the entries should appear as follows:

        oracle.sysman.eml.mntr.emdRepPwd=ece067ffc15edc4f
        oracle.sysman.eml.mntr.emdRepPwdEncrypted=TRUE

## 8.4 Dropping and Recreating the Management Repository

This section provides information about dropping the Management Repository from your existing database and recreating the Management Repository after you install Enterprise Manager.

### 8.4.1 Dropping the Management Repository

To recreate the Management Repository, you first remove the Enterprise Manager schema from your Management Repository database. You accomplish this task using the `-action drop` argument to the `RepManager` script, which is described in the following procedure.

To remove the Management Repository from your database:

1. Locate the `RepManager` script in the following directory of the Oracle Application Server Home where you have installed and deployed the Management Service:

       IAS_HOME/sysman/admin/emdrep/bin

2. At the command prompt, enter the following command:

       $PROMPT> RepManager repository_host repository_port repository_SID
       -sys_password password_for_sys_account -action drop

   In this syntax example:

   - `repository_host` is the machine name where the Management Repository database is located

   - `repository_port` is the Management Repository database listener port address, usually 1521 or 1526

   - `repository_SID` is the Management Repository database system identifier

   - `password_for_sys_account` is the password of the SYS user for the database. For example, `change_on_install`.

   - `-action drop` indicates that you want to drop the Management Repository.

Alternatively, you can use a connect descriptor to identify the database on the RepManager command line. The connect descriptor identifies the host, port, and name of the database using a standard Oracle database syntax.

For example, you can use the connect descriptor as follows to create the Management Repository:

    $PROMPT> ./RepManager -connect "(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)
    (HOST=host1)(PORT=1521)) (CONNECT_DATE=(SERVICE_NAME=servicename)))"
    -sys_password efkl34lmn -action drop

See Also: "Establishing a Connection and Testing the Network" in the Oracle Database Net Services Administrator's Guide for more information about connecting to a database using connect descriptors

### 8.4.2 Recreating the Management Repository

The preferred method for creating the Management Repository is to create the Management Repository during the Enterprise Manager installation procedure, which is performed using Oracle Universal Installer.

See Also: Oracle Enterprise Manager Grid Control Installation and Basic Configuration for information about installing Enterprise Manager

However, if you need to recreate the Management Repository in an existing database, you can use the `RepManager` script, which is installed when you install the Management Service. Refer to the following sections for more information:

- Using the RepManager Script to Create the Management Repository

- Using a Connect Descriptor to Identify the Management Repository Database

#### 8.4.2.1 Using the RepManager Script to Create the Management Repository

To create a Management Repository in an existing database:

1. Review the hardware and software requirements for the Management Repository as described in Oracle Enterprise Manager Grid Control Installation and Basic Configuration. and review the section "Management Repository Deployment Guidelines".

2. Locate the `RepManager` script in the following directory of the Oracle Management Service home directory:

       ORACLE_HOME/sysman/admin/emdrep/bin

3. At the command prompt, enter the following command:

       $PROMPT> ./RepManager repository_host repository_port repository_SID  -sys_password password_for_sys_account -action create

In this syntax example:

- `repository_host` is the machine name where the Management Repository database is located

- `repository_port` is the Management Repository database listener port address, usually 1521 or 1526

- `repository_SID` is the Management Repository database system identifier

- `password_for_sys_account` is the password of the SYS user for the database. For example, `change_on_install`.

Enterprise Manager creates the Management Repository in the database you specified in the command line.

#### 8.4.2.2 Using a Connect Descriptor to Identify the Management Repository Database

Alternatively, you can use a connect descriptor to identify the database on the `RepManager` command line. The connect descriptor identifies the host, port, and name of the database using a standard Oracle database syntax.

For example, you can use the connect descriptor as follows to create the Management Repository:

    $PROMPT> ./RepManager -connect "(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP) (HOST=host1)(PORT=1521)) (CONNECT_DATA=(SERVICE_NAME=servicename)))" -sys_password efkl34lmn -action create

See Also: "Establishing a Connection and Testing the Network" in the Oracle Database Net Services Administrator's Guide for more information about connecting to a database using a connect descriptor

The ability to use a connect string allows you to provide an address list as part of the connection string. The following example shows how you can provide an address list consisting of two listeners as part of the `RepManager` command line. If a listener on one host becomes unavailable, the second listener can still accept incoming requests:

    $PROMPT> ./RepManager -connect "(DESCRIPTION=
    (ADDRESS_LIST=
    (ADDRESS=(PROTOCOL=TCP)(HOST=host1)(PORT=1521)
    (ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521)
    (CONNECT_DATE=(SERVICE_NAME=servicename)))"
    -sys_password efkl34lmn -action create

See Also: Oracle High Availability Architecture and Best Practices "Configuring the Management Service to Use Oracle Net Load Balancing and Failover"

## 8.5 Troubleshooting Management Repository Creation Errors

Oracle Universal Installer creates the Management Repository using a configuration step at the end of the installation process. If the repository configuration tool fails, note the exact error messages displayed in the configuration tools window, wait until the other configuration tools have finished, exit from Universal Installer, and then use the following sections to troubleshoot the problem.

### 8.5.1 Package Body Does Not Exist Error While Creating the Management Repository

If the creation of your Management Repository is interrupted, you may receive the following when you attempt to create or drop the Management Repository at a later time:

    SQL> ERROR:
    ORA-00604: error occurred at recursive SQL level 1
    ORA-04068: existing state of packages has been discarded
    ORA-04067: not executed, package body "SYSMAN.MGMT_USER" does not exist
    ORA-06508: PL/SQL: could not find program unit being called
    ORA-06512: at "SYSMAN.SETEMUSERCONTEXT", line 5
    ORA-06512: at "SYSMAN.CLEAR_EMCONTEXT_ON_LOGOFF", line 4
    ORA-06512: at line 4

To fix this problem, see "General Troubleshooting Techniques for Creating the Management Repository".

### 8.5.2 Server Connection Hung Error While Creating the Management Repository

If you receive an error such as the following when you try to connect to the Management Repository database, you are likely using an unsupported version of the Oracle Database:

    Server Connection Hung

To remedy the problem, upgrade your database to the supported version as described in Oracle Enterprise Manager Grid Control Installation and Basic Configuration.

### 8.5.3 General Troubleshooting Techniques for Creating the Management Repository

If you encounter an error while creating the Management Repository, drop the repository by running the `-drop` argument to the `RepManager` script.

See Also: "Dropping the Management Repository"

If the `RepManager` script drops the repository successfully, try creating the Management Repository again.

If you encounter errors while dropping the Management Repository, do the following:

1. Connect to the database as SYSDBA using SQL*Plus.

2. Check to see if the SYSMAN database user exists in the Management Repository database.

   For example, use the following command to see if the SYSMAN user exists:

       prompt> SELECT username FROM DBA_USERS WHERE username='SYSMAN';

3. If the SYSMAN user exists, drop the user by entering the following SQL*Plus command:

       prompt> DROP USER SYSMAN CASCADE;

4. Check to see if the following triggers exist:

       SYSMAN.EMD_USER_LOGOFF
       SYSMAN.EMD_USER_LOGON

   For example, use the following command to see if the EMD_USER_LOGOFF trigger exists in the database:

       prompt> SELECT trigger_name FROM ALL_TRIGGERS
               WHERE trigger_name='EMD_USER_LOGOFF';

5. If the triggers exist, drop them from the database using the following commands:

       prompt> DROP TRIGGER SYSMAN.EMD_USER_LOGOFF;
       prompt> DROP TRIGGER SYSMAN.EMD_USER_LOGON;

## 8.6 Improving the Login Performance of the Console Home Page

Oracle Enterprise Manager now provides an option that will more quickly display the Console Home page even in a scenario where the Management Repository is very large. Normally, factors such as the number of alerts, errors, policies, and critical patches can contribute to delayed displayed times. Since there is no single factor nor any simple way to scale the SQL or user interface, a simple option flag has been added that removes the following page elements for all users.

When the emoms.properties flag, `LargeRepository=`, is set to true (when normally the default is false), the SQL for the following items is not executed and thus the items will not be displayed on the Console page.

1. Three sections from the Overview Page segment:

   - All Target Alerts

     - Critical

     - Warning

     - Errors

   - All Target Policy Violations

     - Critical

     - Warning

     - Informational

   - All Target Jobs

     - Problem Executions (last 7 days)

     - Suspended Executions (last 7 days)

2. The page segment which includes Security Patch Violations and Critical Patch Advisories.

The Deployment Summary section would move up to fill in the vacated space.